CONTINUOUS AND RANDOM VAPNIK-CHERVONENKIS CLASSES 



ITAI BEN YAACOV 

Abstract. Nous demontrons que si T est une theorie dependante, sa randomisee de 
Keisler T R Test aussi. 

Pour faire cela nous generalisons la notion d'une classe de Vapnik-Chervonenkis a des 
families de fonctions a valeurs dans [0, 1] (une classe de Vapnik-Chervonenkis continue), 
et nous caracterisons les families de fonctions ayant cette propriete par la vitesse de 
croissance de la largeur moyenne d'une famille de compacts convexes associes. 

In this paper we answer a question lying at the intersection of two currently active 
research themes in model theory. 

The first theme is that of dependent theories, i.e., first order theories which do no 
possess the independence property, first defined by Shelah |She71j . A formula (p(x,y) is 
said to have the independence property, or to be independent, in a theory T if for every 
n one can find a model M. 1= T, hi 6 M for i < n, and a w e M for w C n, such that 
M. N (f(a w , bi) i G w. The theory T has the independence property if at least one 
formula has it in T; equivalently, a theory T is dependent if every formula is. A stable 
theory is necessarily dependent. (More generally, a theory T is unstable if and only if 
it is independent or has the strict order property. See |Poi85j Theoreme 12.38] for the 
proof in classical first order logic. It can be adapted easily to continuous logic following 
standard translation methods.) 

The recent use of properties of dependent theories for the solution of the so-called Pillay 
Conjecture in [HPP] earned them a considerable increase in general interest. It should be 
pointed out that some refer to dependent theories as NIP (Non Independence Property) 
theories. Since non-NIP theories are quite "wild", research in this area concentrates on 
theories which are not non-NIP. 

The second theme of research is that of continuous logic and metric structures. Study- 
ing metric structures using a model theoretic approach dates back to Henson [Hen76j 
and Krivine and Maurey |KM81j . Continuous first order logic was much more recently 
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introduced in [BUj as a formalism for this study, shifting the point of view much closer 
to classical first order logic. The independence property has a natural analogue for met- 
ric structures, and one may speak of dependent continuous theories. In contrast with 
the body of work concerning classical dependent theories, to the best of our knowledge 
dependent continuous theories have hardly been studied to date. 

The intersection of these two themes in which we are interested stems from H. Jerome 
Keisler's randomisation construction. This first appeared in [Kei99] . where to every 
(complete) first order theory T he associated the first order theory of spaces of random 
variables in models of T. Since classical first order logic is not entirely adequate for the 
treatment of spaces of random variables, which are metric by nature, this construction 
was subsequently improved to produce for every theory T the continuous theory T R of 
spaces of random variables taking values in models of T (see |BKj ). While we shall not 
go through the details of the construction, we shall point the main properties of T R in 
Section [51 The author has shown that 

(i) The randomisation of a stable theory is stable ( [BKj . in preparation). 

(ii) On the other hand, the randomisation of a simple unstable theory is not simple. 
More generally, the randomisation of an independent theory cannot be simple, 
and is generally wild. 

In other words, independent theories are somehow wild with respect to randomisation 
(even if they do satisfy other tameness properties such as simplicity), while stable the- 
ories are tame. It is natural to ask whether the dividing line for tame randomisation 
lies precisely between dependent and independent theories. A more precise instance of 
this question would be, is the randomisation of a dependent theory dependent? While 
Keisler's construction was only stated for a complete classical theory T it can be carried 
out just as well for an arbitrary continuous theory T (with some minor technical changes) 
and the question can also be posed when T is continuous. 

The article [HPPJ mentioned above relates dependent theories and probability mea- 
sures on types, i.e., with types in the randomised theory. This suggested that dependent 
theories should be tame with respect to randomisation, so the answer to the question 
above should be positive. (To our best recollection this was first conjectured by Anand 
Pillay in the AIM workshop on Model Theory of Metric Structures, September 2006). 

In order to give a positive answer we shall consider some purely combinatorial aspects 
of the independence property, observed by Shelah |She71] and independently by Vapnik 
and Chervonenkis [VC71j and seek to prove they extend to the continuous setting. (The 
connection between dependent theories and the work of Vapnik and Chervonenkis was 
pointed out by Laskowski |Las92j .) Doing so we will need a new means for measuring 
the size of a set, as merely counting points will no longer do. The Gaussian mean width 
turns out to serve our purposes quite well (the Lebesgue measure of the set once inflated 
a little will also be useful, but to a much lesser extent). The Gaussian mean width 
commutes, in a sense, with the randomisation construction, and it follows painlessly 
that the randomisation of a dependent relation is again a dependent relation. Thus the 
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technical core of this paper has nothing to do with model theory and deals rather with 
combinatorics and the geometry of convex compacts. 

Section [1] consists of a few basic facts regarding convex compacts in M n and their mean 
width. 

The combinatorial core of the article is in Section [2] where continuous Vapnik- 
Chervonenkis classes and dependent relations are characterised via the growth rate of 
the mean width of an associated family of sets. 

Section [3] consists of a technical interlude where we prove that continuous combinations 
of dependent relations are dependent. 

In Section H] we consider random dependent relations. Using the mean width criterion 
from Section [2] we show that if a random family of functions is uniformly dependent then 
its expectation is dependent as well. 

The proper model theoretic contents of this paper is restricted to Section [5j We define 
dependent theories and the randomisation of a theory. The main theorem, asserting that 
the randomisation of a dependent theory is dependent, follows easily from earlier results. 
We also extend to continuous logic a classical result saying that in order to verify that a 
theory is dependent it suffices to verify that every formula (p(x,y) is dependent where x 
is a single variable. 



This section contains few properties of the mean width function. The author is much 
indebted to Guillaume Aubrun for having introduced him to this notion and its prop- 
erties. All the results presented here are easy to verify and are either folklore (see for 
example (AS06J) or (in the case of integrals of convex compacts, as far as we know) minor 
generalisations thereof. 

Let KPa bounded set. For y G M. n define fiA(y) = sup a . gj4 (x, y). As a function of 
y, Iia is positively homogeneous and sub-additive, and thus in particular convex. 

Let u G S n ~ l . The real numbers t\ = }ia{u) and t 2 = —hA(—u) are then minimal and 
maximal, respectively, so that t 2 < (%, u) < t\ for all x G A, i.e., such that A lies between 
the two hyperplanes t 2 u + u 1 - and tiu + u L . The width of A in the direction u G S 11 ^ 1 is 
therefore defined to be w(A, u) = tiA^u) + Iia(—u). 

Let K = Conv(A) be the closed convex envelope of A, i.e., the intersection of all closed 
half-spaces containing A. Then hx = h,A and: 



We may thus identify a convex compact K C R n with hx '■ S n 1 — > R. In this case the 
supremum in the definition of hx is attained at an extremal point of K. 



1. Facts regarding convex compacts and mean width 




4 



ITAI BEN YAACOV 



It is easy to observe that the function h K {u) is monotone, positively homogeneous and 
additive in K, i.e., K C K' =^ h K {u) < h K i(u), h a K(u) = ahxiu) for a > and 

hK+K'(u) — max (x,u) = max (y + z,u) 

x£K+K' y£K,z£K' 

= max(w,w) + max(z, w) = hx(u) + hx'(u). 

y£K zeK' 

Let (X, 23, /i) be a measure space, K a mapping from X to the space of convex compacts 
in R™. Say that K is measurable (respectively, integrable) if uo \— > /iK(aj)( M ) is for all u G S 1 . 
Notice that u \— > /ik(w)( u ) is o(K(cj))-Lipschitz where &(if) = max{/ix(«): u G S^ -1 }. 
Thus, if hit(u) is measurable for all u in some dense (and possibly countable) subset of 
S' n ~ 1 then 6(K) is measurable and thus K is. If K is integrable define: 



h(u) = J hf^{u) d/j,, 
K= [Kd[i= p| {x: (x,u) <h(u)}. 

J -.^Clr, — 1 



ues n 

Clearly if is a convex compact, and if x: X — > R" satisfies x(u;) G K(u;) a.e. then 
f xa> G f K d/i. We claim furthermore that /ix = /i. Indeed, it is clear by definition 
of K that /ix < /i. Conversely, given u G S*™ -1 we may complete it to an orthonormal 
basis -u — u,Ui, . . . ,u n . For each u G X there is a unique x w G K(c<j) such that the 
tuple ((x u , Uq), . . . , (x u , M n -i)) is maximal in lexicographical order (among all x G K(o>)). 
In particular (x u ,u) = hn^^u). Moreover, the mapping x: uj i— > rr^ is measurable, 
x = f x.d/1 G if and hx(u) > (x,u) = h(u). Thus /ix = h as required. 
The mean widi/i of if is classically defined as: 

w(K) — / w(K,u)dfi = 2 / h K (u)da, 
Js"- 1 Js™- 1 

where a is the normalised Lebesgue measure on the sphere. 

Lemma 1.1. The mean width is a monotone, additive, positively homogeneous func- 
tion of compact convex subsets o/R™. Moreover, if K is an integrable family of convex 
compacts then 



w(^jKdf?j = J w(K)dfi. 



Proof. Monotonicity of w follows from monotonicity (in K) of hx- Additivity and positive 
homogeneity are special cases of the summability which follows from earlier observations 
via Fubini's Theorem. Bi.i 

As it happens it will be easier to calculate the following variant of the mean width: 

Definition 1.2. The Gaussian mean width of a convex compact K is defined as 

w G (K) = E[w(K,G n )} = 2E[h K (G n )}, 
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where G n ~ N(0, I n ) (i.e., G n = (g , g n _{) where g , . . . , g n _i are independent ran- 
dom variables, <?j ~ N(0, 1)). 



Since the distribution of N(0,I n ) is rotation-invariant, the random variables ||Cr-||_ 
and phr are independent. Let 7„ = E[||G n || 2 ]. We obtain: 

w G (K) = E[\\G n \\ 2 w(K,G n /\\G n \\ 2 )} = E[\\G n \\ 2 ]E[w(K,G n /\\G n \\ 2 )} = ln w(K). 
One can further calculate that 

r(f) 



n = V 7 ^- 



r(|) ' 

As T is log-convex we obtain: 



r(f + i) rn __ 



Whence: 



- 1 < In < Vn 

Thus for example, if B n is the unit ball in M. n then 

w G (B n ) = ln w{B n ) = 2 ln « 2v^. 

Lemma 1.3. T/ie Gaussian mean width is a monotone, additive, positively homogeneous, 
function of compact convex subsets ofW 1 , and for an integrable family K: wq (f K c?//J = 
/ w G (K) dfx. 

Proof. Follows from Lemma [1.11 (or is proved identically). Bi.3 

Let us calculate the mean width of the cube [—1,1]™. The maximum 
hxiy) — max aje[-i,i]™( x ' v) i s & l wa y s attained at an extremal point, i.e., h K {y) = 
max ie{ _ 1) i } „(x,?/) = \\y\\i. Thus: 

w G ([-l, l} n ) = 2E[\\G n \\ 1 ] = 2nE[\G 1 \] = 2m/-. 

V 7T 

Thus, for e > we have: 

(1) t_ o ([0,e] B ) = enj|. 
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2. Fuzzy and continuous Vapnik-Chervonenkis classes 

Let us start with a few reminders regarding the Vapnik-Chervonenkis classes. We shall 
follow Chapter 5 of van den Dries [vdD98j . 

Let us fix a set X and a family of subsets C C V(X). Recall that [X] n denotes the 
collection of all subsets of X of size n, and let V^(X) = [j n<UJ [X} n denote the collection 
of finite subsets of X. For F G V s (X) and n < uj let: 

CnF = {CnF-. C eC}, 

f c {n) = max{|C n F\ : F G [X] n }. 

Clearly, fc(n) < 2 n . Define the Vapnik-Chervonenkis index of C, denoted VC(C), to 
be the minimal d such that /c(rf) < 2 d , or infinity if no such d exists. If VC(C) < oo 
then C is a Vapnik-Chervonenkis class. 

Let Pd(x) = X]fc<d (fc) e QNj observing this is a polynomial of degree d — 1. 

Fact 2.1. J/d = 1/C7(C) < oo &en / c (n) < p d {n) for all n. 

This can be viewed as a dichotomy result: either \C fl F\ is maximal (given |F|) 
for arbitrarily large finite F C X, or it is always quite small (polynomial rather than 
exponential). It follows immediately from the following. 

Fact 2.2. Let F be a finite set, n = \F\, and say T> C V(F) is such that \T>\ > Pd{n). 
Then F admits a subset E C F, \E\ — d such that [D fl E\ = 2 d . 

See |vdD98l Chapter 5] for the proof, which is attributed independently to Shelah 
[She71j and to Vapnik and Chervonenkis |VC71j . This will also follow as a special case 
of a result we prove below. 

Let us now add a minor twist to the setting, whose motivation will become clear later 
on. We allow the class C to contain fuzzy subsets of X, i.e., objects C such that for each 
x G X at most one of x G C or x £ C holds, but possibly neither (in which case it is 
not known whether x belongs to C or not). This can be formalised by pair C = (Ci, C2) 
where Ci,C 2 C X are disjoint, C\ = {x G X : x G C}, C 2 = {x G X : x £ C}. 

Let CCI denote that C is a fuzzy subset of X and let P{X) denote the collection 
of fuzzy subsets. If F C X, we say that C determines a subset of F if for all x G F one 
of x G C or x ^ C does hold, in which case we define C fl F as usual, and otherwise we 
define F fl C = * . We may then define 

CnF = {CnF-. CgC}\{*}, 

f c (n) = max{|C n F\ : F G [X]"}. 

Thus C PI F is the collection of all subsets of F which members of C determine. Vapnik- 
Chervonenkis classes of fuzzy subsets of X and the corresponding index are defined as 
above, and the standard proofs of Fact 12.11 and of Fact 12.21 hold verbatim. 

Our source for classes of fuzzy subsets of X will be the following. Let Q C [0,1]^ be 
a collection of functions from X to [0, 1]. For < r < s < 1 and q G Q we define a 
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fuzzy set q r ^ s C X as follows: x G q r ^ s if g(a;) > s, x q r>s if < r, and it is unknown 
whether x belongs to q r>s or not if r < q(x) < s. We define Q r ^ s = {q r>s '- <? G Q} ^ 
We say that Q is a Vapnik-Chervonenkis class if Q ryS is for every < r < s < 1. Of 
course the index may vary with r, s. However an easy argument shows that if Q is a 
Vapnik-Chervonenkis class then for every e > there exists < uj which is an upper 
bound for the Vapnik-Chervonenkis indexes of the classes Q r , r +e as r varies in [0, 1 — e\. 
Notice that in the original case where C C V(X), if Q = {xc : C G C} is the collection 
of characteristic functions of members of C then Q TyS = C for every < r < s < 1, so the 
subset case is a special case of the function case. 

Definition 2.3. Let < r\ < < 1 be given for % < n and let A C [0, 1]™. We say 
that A determines a subset w C n between f and s if there is a point a G A such that 
i <E w =^ cij > Sj and i w < r\ for all i < n. In case r\ = r and Sj = s for all 

i < n we say that A determines w between r and s. 

We say that A determines a d-dimensional e-box If e > 0, < n, and there are 
i < . . . < id-i < n and f G [0, 1 — e] d such that 7Tj(A) C [0, l] d determines every subset 
of d between f and f + e. 

Finally for e > we say that A determines a strict d-dimensional e-box if it determines 
a o?-dimensional e'-box for some e 1 > e. 

Thus if F — {x±, . . . ,x n } C X then <5 r , s H F is in bijection with the subsets of n 
determined by Q{x) between r and s. 

Let us now relate this to the previous section. Let again Q C [0, l] x be a collection of 
functions. For a tuple x G X n and g G Q define: 



Lemma 2.4. //A C [0, 1]™ determines an n- dimensional e-box then w G (A) > enJ-. If 



Proof. It suffices to prove the first assertion. In this case there are f G [0, 1 — e] n and for 
every w C n there is a w E A such that ««,(«) > r + e if i G w and a w (i) < r otherwise. 
Thus A D (a w : w C n) D n + e] = n + [0, e]™. It follows that 



In other words, if A determines an n-dimensional e-box then Conv(A) contains a set 
of the form f + [0, e] n . The converse does not hold in general. 



q(x) = (q(x ),...,q(x n )) G [0, l] n , 

Q(x) = (q(x): q G Q), 
g Q {n) = sup{w G (Q(x)) : x G X n }. 




A determines a strict n-dimensional e-box then wg(A) > en 





'2.4 



Proposition 2.5. If Q C [0,1] 



and lim = toera Q zs a Vapnik-Chervonenkis 



class. 
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Moreover, for any function gin) such that lim = and any e > there is d(g, e) < 
to such that for any Q C [0, 1} X , if < g then d(g,e) > VC(Q rjr+£ ) for allO < r < 1 — s. 



Proof. Let g = gq and e > be given, and find n such that < ey |. We claim that 
d(<7, e) = n will do. 

Indeed, assume not. Then there are r £ [0, 1 — e\ and F = {xo, . . . , x n _i} C X such 
that |Q rjr+e n F\ = 2 n , i.e., such that determines an n-dimensional e-box. By 

Lemma [2.41 we have g(n) > gqin) > Wg{Q{x)) > £nJ^ a contradiction. B 2 .5 



For the converse a little more work is required. Let 7r : M. n — > MJ 1 1 be the projection 
on the first n-1 coordinates. For ACM™ and a G R let A< a = in (M n_1 x ]— oo, a]), 
y4 >a = An (R n_1 x ]a, +oo[). 

Let A denote the Lebesgue measure. 

Lemma 2.6. Let AC [0,1 + l] n fre a 5ore/ set, X(A) > £ d p d {n). Then at least one of the 
following holds: 

(i) X(irA)> £ d p d {n-l). 

(ii) There is a G [0,£+ 1] sucft that A(vrA< a n 7rA >a+1 ) > f^pd-iC™ - !)■ 
Proo/. For x G [0, £ + If" 1 let 

A' = {(£, a) G [0, £ + If" 1 x [0, £] : x G ixA< a n 7rA >a+1 } 



/aO) = y X^foy) /a'C^O = J XA'(x,y) dy. 

Notice that /^'(x) + XttA^) > /a(^), integrating which yields: 

X(A') + X(7r(A)) > X(A) > £ d p d (n). 

Recall that p d {n) = p d -±(n — 1) +Pd{ n — 1) and assume that the first case fails, i.e., that 
X(nA) < £ d p d {n - 1). Then: 

X(A') + £ d p d [n - 1) > £ d p d {n) = - 1) + e d Pd(n - 1) 

whereby: 

X(A')>£p d ^(n-l). 

Then there is a such that A({x: (x, a) G A'}) > p d -\{n — 1), which is precisely the second 
case. B 2 .6 

Lemma 2.7. Lei AC [0,£+ l] n be a Borel set, X(A) > £ d p d (n). Then A determines a 
strict d- dimensional 1-box. 

Proof. Follows immediately by induction on n using the previous Lemma for the induction 
step. M 2 .7 
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Lemma 2.8. Let A C [0, l] n and c > 0, e > be such that 

X(A + [0, c] n ) > (c + e) n - d (l - s) d Pd (n). 
Then A determines a strict d- dimensional e-box. 

Proof. Let £ = g, so i + 1 = ±±§ . Let 5 = (c + e) -1 (X + [0, c] n ). Then BC[0,f+ 1]" 
and A(-B) > £ d pd(n). By Lemma 12.71 i? determines a strict rf-dimensional 1-box. Thus 
A + [0, c] n determines a strict (i-dimensional (c + e)-box, and A determines a strict d- 
dimensional e-box. M 2 .s 



In case A C {0, l} n then X(A + [0, l] n ) = \A\. If in addition \A\ > p d (n) = (1 + 
0)"~ d (l — 0) d prf(n) then A determines a (strict) (i-dimensional 0-box, i.e., an e-box for 
some arbitrarily small e > 0. But given that A C {0, l} d this is only possible if A 
determines a (i-dimensional 1-box. Thus Fact 12.21 follows as a special case of Lemma [2.81 

Now let us show that if \(A + [0, c] n ) is small then A is small in a different way, namely 
has small Gaussian mean width. 

Lemma 2.9. Let A C [0, If, c> 0. T/jen 

u>g(A) < (1 + c)^2n\og(X(A+[0,c} n )/c n ). 

Proof. Let us first observe that if <p : W l — > R is a linear functional then the convexity of 
the exponential function implies: 

exp^rr)) = exp ( / <p(y) (2c)- n d\(y)) 

<(2c)-» / expfo(y))dA(y). 

</:z+[-c,c] n 

Let A' = 2A-1C [-l,l] n and 5' = A' + [-c,c] n = 2(A+ [0,c\ n ) - (1 + c). Then by the 
previous observation we have: 

sup{exp(y?(x)): x E A'} < (2c)~ n / exp(^(x)) d\(x). 

Jb> 

Let (3 > be an arbitrary parameter for the time being. For a fixed x E W 1 we have 
P(x,G n ) ~ N(0, (3 2 \\x\\ 2 ), and a straightforward calculation yields E[exp(/3(x, G n ))] = 
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exp(/? 2 1| x || 2 /2) . Using concavity of the logarithm we obtain: 

w G (A') = 2E [sup {{x, G n ) : x G A'}] 
2 

= -E [log (sup {exp(3(x, G n )) : x G A'})] 
< | log ( E 



(2cY n / eM3{x } G n ))d\(x) 



B> 



| log ( (2c)- n / E[exp(/5(x,G n ))] dA(;r; 



2, / /0 .-n /" „„„^ 2 |kl! 2 



\og((2c)- n J exp 



/? V 7b' V 2 

'/3 2 (l + c) 2 n 



<|log f(2c)-"A(fl') exp ( 



2\og{\{B')/{2 C ) , „, „, 

P 



2 

+ 3(l + c)' z n. 



Minimum is attained when 3 = V 21o ^ A (^ )/( 2c ) ) ^ substituting we obtain: 

u; G (A') < 2(1 + c)v/2nlog(A(5')/(2c)«). 
Finally, w G (A) = w G (A')/2 and X(A+ [0, c] n ) = \(B')/2 n , whence the desired inequality. 

^2.9 



Lemma 2.10. Lei A C [0, 1]" be finite, \A\ = N. Then w G (A) < ^2n \ogN . 

Proof. For c small enough we have X(A + [0, c] n ) = Nc n , so w G (A) < (1 + c) \/2n log 
and thus wg(^4) < log N. B2.10 

Our proof of Lemma 12.91 is based on a direct argument due to M. Talagrand for 
Lemma 12.101 

Theorem 2.11. Let Q C [0, l] x 6e a collection of functions. Then the following are 
equivalent: 

(i) Q is a Vapnik-Chervonenkis class. 

(ii) For every e > i/iere zs d siic/i i/iat for every x G X d ; does not determine 
a d- dimensional e-box. 

fiii) l im 9£M = . 



Proof. For (i) ==>- (ii) we shall prove the contra-positive. So assume that for some e > 
this fails, i.e., for all d there are x G X d , r , . . . , r^-i and {q w : w C dj C Q satisfying 
q w (xi) < Ti if z G w and q w {xi) > r j + e if i ^ w. Thus there must be a subset of of size 
at least d! = \de/2] which are at distance at most e/2 from one another, and we might 
as well assume these are r < r\ < . . . < 7>-i < r + e/2 = r. For i < d! we q w {xj) < r 
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if i G w and q w {xj) > r + e/2 if i ^ w. This works for arbitrarily large d, and thus 
for arbitrarily large d! . Thus Q is not a Vapnik-Chervonenkis class. (And considering 
d! = [de/m\ we can get q w ( xj) > r + e(l - ^).) 
Let us now show (i 



in 



Let us fix e > 0, and let d be as in the hypothesis. By 



Lemma [2.81 and Lemma [2.91 we have for all c > and x G X n \ 

X(Q(x) + [0, c] n ) < (c + e) n - d (l - e) d Pd (n), 
w G {Q{x)) < (1 + cW2nlog{\{Q{x) + [0,c] n )/d« 

Whereby: 



9q{n) < (1 + c 
9Q(n 



\ 



2n log 1 + 



C + £ 



< 1 + C 



n 



\ 



21og(l + -) + -lo 

c/ n 




1 - e 

C + £ 



As n goes to infinity the second term under the root disappears. In addition we have 
log(l + e/c) < e/c, and we obtain: lim < (1 + c) y^. Minimum is reached when 

c = 1 in which case lim gg(n ^ < v^8i- This holds for every e > 0, whereby lim — o 
as desired. 

) was proved in Proposition 12.51 



in. 



■2.11 



Notice that the proof also tells us in fact something more precise: 

Corollary 2.12. Assume that lim 9Q ^ = C > 0. Then for some r the class Q r , r +c 2 /8 is 
not a Vapnik-Chervonenkis class. 

In case Q C {0, 1} X , i.e., for collection of characteristic functions, a box if exists has 
size one, so we can get better bounds . 

Proposition 2.13. Let C C T(X), Q = {xc- C G C} C [0, T/ien g Q (n) < 

\j2n \ogpd(n) and for n big enough gq(n) < \j2dn logn ; where d = VC(C). 

Proof. Let d = VC(C) < oo. For every x G X n we have < fc{ n ) ^ Pd(n) by 

Fact 12.11 For n large enough we have Pd(n) < n d and by Lemma [2. 101 



9Q{n) < \/2n \ogp d (n) < ^j2dn\ogn. 



■2.13 



We can now switch to a more symmetric situation. Let X and Y be two sets, S C 
X x y. For x G X let 5 X = {y G G 5} and for y & Y let ^ = {x G 

X : (x, ?/) G S 1 }. Thus S gives rise to two families of subsets S Y = {S y : y G Y} C V(X) 
and ^ x = {£r : x G A} C P(y). 
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Similarly, if S □ X x Y we may define S 9 C I by i G S' 2 ' •<=>■ (x, y) & S and 
x S y -<=>• (x, y) ^ iS. Continuing as above we obtain two families of fuzzy subsets 
S Y C P(X) and S x C P(F). 

Fact 2.14. Let S ^ X x Y . Then Sx is a Vapnik-Chervonenkis class if and only if S Y 
is, in which case VC(Sx) < 2 VC ^ and vice versa. 
We say in this case that S is a dependent relation. 

Proof. In case S C X x Y this is proved in [vdD98, Chapter 5]. The case of a fuzzy 
relation, while not considered there, is identical. B2.14 

Finally, a function ip: X x Y — > [0, 1] gives rise to two families of functions ip Y = 
{ v y-.yeY} = {(p(-, y):yeY}C [0, 1} X and similarly = x e X} C [0, l] y . 

Proposition 2.15. Let X and Y be sets, ip: X x Y — ► [0, 1] any function. Then (p Y is 
a Vapnik-Chervonenkis class if and only if (fx is. 
In that case we say that ip is dependent. 

Proof. For < r < s < 1 define (p rs C X x K as usual. Then {<p> ryi )x — { l Px)r,s is a 
Vapnik-Chervonenkis class if and only if (ip r ,s) Y — {f Y )r,s is. '2.15 

Lemma 2.16. A uniform limit of dependent functions is dependent. 

Proof. Let ip n : X x Y — > [0, 1] be dependent converging uniformly to ip. Assume ip is 
independent, so say ip r , r +3e is independent for some e > and r 6 [0,1- 3e]. Let n 
be large enough such that \<p — ip n \ < e. Then (ip n )r+e,r+2e is independent, contrary to 
hypothesis. B2.16 

3. Crushing convex compacts 

Let K C R n be a convex compact, u G S*™ -1 a fixed direction vector. We would like to 
construct a new convex compact K u by crushing all points below the hyperplane u 1 - to 
the hyperplane. Define the two half spaces and a mapping S : W 1 — > W 1 as follows: 

H + = {xe R n : (x,u) > 0}, H ={xe W l : (x,u) < 0}, 




We then let 

K u = Conv(S(K)) = Conv((fT n H + ) U P U ±(K H H~)). 

We would like to show that wg(K u ) < wg(K). 

Lemma 3.1. Let K , u and K u be as above. Ifhxi—u) < then K = K u . If hx{—u) > 
then we have for y e W n , y' = P u ±(y): 

h K u (y) = m&x(h K (y), h K (P u ±(y))) y e H + , 

h Ku (y) < min{h K (z): z G [y,P u ±(y)]} y G H . 
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Proof. If h K (-u) < then K C H+ and S(K) = K. Consider the case h K (-u) > 0. In 
that case clearly Iik u {u) = and hx u agrees with on u L . Let us also observe that if 
y G W 1 then the hK u {y) = (x,y) for some extremal point x G K u , in which case we have 
in fact x G S(K). Thus we always have h,K u (y) = (S(x),y), x G K. 

Let us consider the case where y G H + , i.e., y = y' + \u where y' _L u and A > 0. 
Say hxiy) = (x,y), x G K. Then (S(x),y) > (x,y) and thus Hk u > hx(y)- Since 
is sub-additive we also have hx u (y) = h Ku (y' + Am) + Am) > h Ku {y'). Thus 

hK u {y) > niax(/ift;(2/), hxiy'))- On the other hand, we know that h.K u (y) = (S(x),y) for 
some If rr G H + then ^(y) < ^(y). If x G if" then (S(x),y) = (S(x),y') = 

(x,y') so h Ku (y) < h K (y'). Either way h Ku (y) < max(h K (y), h K (y')) and the first case 
is proved. 

Now assume y G H~ . Let us make first some general observations. First, if hx u {y) = 
(S(x),y), x G K, then (S(x),y) < (x,y) whereby hx u (y) < h K (y). Now write y = y' — Xu 
where y' _L u and A > 0. Let z G [y, y'] C H~ , i.e., z = y' — fiu for u G [0, A]. Then 
hK u {y) < hK u (z) + hK u (—{X — fi)u) = h,K u (z) < h-K^z)- We have thus shown that 
/ix„(y) < min{/i^(z): z G [y',y]}. B 3 .i 

Can the second inequality be improved to an equality? Either way, the inequalities we 
have suffice to prove: 

Lemma 3.2. Let K and K u be as above, y' G u 1 - and A > 0. Then 

hK u {y' + + hi< u {y' - Aw) < h K (y' + Am) + /i x (y' - Am). 

Proof. Consider the mapping s(t) = hx u (y' + tu), which we know to be convex. If 
s{0) < s(A) then h,K u (y' + Am) = h K (y' + Am), and we already know that h Ku (y' — Am) < 
h K (y' ~ Am). 

If s(0) > s(A) then h Ku (y' + Am) = h K (y'). By convexity of s it must be decreasing for 
all t < 0, so in particular 

h Ku {y - Am) < min{s(t) : t G [-A, 0]} = s(0) = ^(y')- 

Thus: 

/i Xu (y' + Am) + h Ku (y' - Am) < h K (2y') < h K (y' + Am) + h K (y' - Am). B 3 .2 

Proposition 3.3. Let K and K u be as above. Then w(K u ) < w(K) and wg(K u ) < 
w G (K). 
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Proof. It will be enough to prove the first inequality. For y G S^ 1 let y' always denote 
P u ±(y) and A = \ {y, u)\. We have: 



2w(K u ) = 2l w{K u ,y) da{y) 

{w(K u , y' + Xu) + w(K u , y' - Xu)) da(y) 



i {h Ku (y' + Xu) + h Ku (y' - Xu) 

+ h Ku (-y' + A«) + ^(-2/' - An)) do-(y) 

< / (h K (y' + Xu) + h K (y' - Xu) 

J gn — 1 

+ M~f' + A«) + - Xu)) da{y) 

= ... = 2w{K). B 3 .3 

Now let K C ]R n be a convex compact, (e^: i < n) the canonical base, and define 
tf + = (. . . (K eo ) ei . . 

= Conv ((x V 0, ... , x n _i V 0) : i 6 K) . 

Corollary 3.4. Let i(Cl" be a convex compact. Then u>g{K + ) < Wg(K). 

We remind the reader that for x, y G [0, 1] we define ->x = 1 — x G [0, 1] and x — y — 
max(x — ?/, 0) G [0, 1]. Moreover, for every n > 1, the family of functions [0, l] n — > [0, 1] 
one can construct with the three operations x — ?/} is dense in the space all 

continuous functions from [0, l] n to [0, 1] (see [BU] ) . 

Corollary 3.5. Let X andY be sets, <p,i/j: Ix7-> [0, 1]. Then gt^y = g^r , 9(ip/2) Y = 
Ig^Y and g^^y < g^r + g^v . 

Thus, if (f and ip are Vapnik-Chervonenkis classes then so are -up, \p> and p — ip. 

Proof. Clearly g^y = g^Y, g^/ 2 y = \g^ ■ We are left with g&^y < g^r + g^r. 

Consider the function ip — ip: X X Y — > [— 1, 1], and observe that for x G X n we have 
O - ip) Y (x) = ({cp - ip) Y (x)) + C [0, l] n . We thus have: 

w G {(tp - iP) Y (x)) = w G (fe - 4>) Y (x)) + ) < w G {{ip - 4>) Y {x)) . 
On the other hand we also have (<p — ip) C ip Y — ip Y C [0, l] x , and for x G X n : 
w G {{.V ~ *P) Y (x)) < w G {<p Y (x) - ip Y (x)) = w G {w Y (x)) + w G {ip Y (x)) . 
Thus w G ((ip ip) Y {%)) < w G {<p Y (x))+w G {ip Y (x)), whereby g(^y ^g^+g^v. M 3 , 5 

Lemma 3.6. Let X and Y be sets, ip, ip : X X Y — > [0, 1] dependent. Then -up, and 
p — ip are dependent as well. 
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Proof. By Corollary 13.51 M 3 G 

Proposition 3.7. Let X and Y be sets, ip n : X x Y — > [0, 1] dependent functions for 
n < uj, and let ip '■ [0, 1]^ — > [0, 1] be an arbitrary continuous function. Then ipo(ip n ) ; X x 
Y — ► [0, 1] is dependent. 

Proof. By results in |BU] one can approximate ip uniformly with expressions written with 
-i, | and — . Such expressions in the (p n are dependent by Lemma Thus ip o (ip n ) is a 
uniform limit of dependent functions, and is therefore dependent by Lemma [2. 161 ^3.7 



4. Random dependent relations and functions 

In this section X and Y will be sets as before. However, we will be interested here in 
dependent relations and functions onlxY which may vary (randomly). 

Let Q be an arbitrary set for the time being. A family of relations on X x Y, indexed 
by Q, can be viewed as a relation S C Q x X x Y. For every wedwe obtain a relation 
S u C X x Y and we may view S equivalently as a function S : Q — > ViX x Y). Similarly, 
a family of [0, l]-valued functions onlxy will be given as <p : Q x X x F — ► [0, 1] or 
equivalently as <p: ft — > [0, l] Xxy sending ii)h^ = •, •). The usual passage from S 
to its characteristic function xs commutes with these equivalent presentations. 

We say that such a family S = {S^ : uj G f2} is uniformly dependent if there is d — d(S) 
such that VC((S W ) Y ) < d for every w G O. Similarly a family = {y^: cj G Q} is 
uniformly dependent if for every e > there is ei = d(<^, e) such that V^C ((</?w)[r,r+e]) < d 
for every r G [0, 1 — e] and uj G fi. Clearly S 1 is uniformly dependent if and only if xs is. 

It follows from the proof of Theorem 12. lll that (p = {ip^ : uj G Q} is uniformly dependent 
if and only if there is a function g : N — > R such that lim = and g^y < g for every 
w. Indeed, in case <p is uniformly dependent then for every uj, n and e we obtain: 



g v Y{n) <2n\\2e + — - — log' 1 



n \ 1 + e J n 

Then a function g as desired can be obtained by: 



g(n) = inf 2nJ2e + ^ log + lo S 



o<e<i y n \ 1 + £/ n 

Let us now consider random relations and functions on X x V. We fix a probability 
space (fi, 03, //). From now on we will only consider families 5 or such that for (x, y) G 
1x7 the event {cu : (x, y) G S^} or the function uj 1— ► ^(x, y) are measurable. We may 
then define functions ¥[S], E[np] : X x F -> [0, 1] by 

P[5](x,y) =P[(x,y) G 5], E[p](a;,y) =E[^(x, 2 /)]. 

If S is measurable then so is xs which is given by {xs)w — X(S W ) an d then E[xs] = P[5]- 
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Theorem 4.1. Let X, Y be countable sets, p>^\ X x Y — > [0,1] a random family of 
functions on X x Y . Then g^^v < Efg^y] (and the latter is measurable). 
In particular, if if is uniformly dependent then E[y>] is dependent. 

Proof. Let us fix n and let x G X n . Define 

K 5 (u;) = C^(^(x)) C [0,1]™ 

Each K £ (a>) is a convex compact and 

g v Y(n) = sup w G (Kx{u})). 

Since K is assumed countable the family K 5 is measurable for every x G X. It is 
moreover bounded and therefore integrable. Since X is also assumed countable the 
function uj \— > g^y (n) is measurable as well. For a fixed tuple x we have E[</>] y (x) C E[K^]. 
Thus 

w G (E[^] y (a;)) < w G (E[K] a ) = E[«j g (K s )] < E[^,v(n)]. 

It follows that gE[p]Y < E[g^y] as desired. 

If <y9 is uniformly dependent then there is g: N — > K such that — > and g > g v Y 
for every uj G fl. Then <7em y < <? as wen an d by Theorem 12 . 1 1 1 E [up] is dependent. B^i 

Corollary 4.2. Let X, Y be sets, p> = {y? w : uj £ Q} a measurable family of uniformly 
dependent functions. Then E[y>] : I x V -» [0, 1] zs dependent. 

Proof. If not then this is witnesses on countable subsets Xq C X and Y"o ^ Y, contra- 
dicting Theorem 14.11 

Corollary 4.3. Lei X ; Y be sets, S = {S^: uj G Q} a measurable family of uniformly 
dependent relations. Then F[S] : X x Y — > [0, 1] zs dependent. 

Proof. Apply Corollary 14.21 to xs- B 4 . 3 

5. Dependent and randomised theories 

In this final section we settle the model theoretic problem which motivated the present 
study. This consists mostly of translating consequences of previous results to the model 
theoretic setting. In order to avoid blowing this section up disproportionately with a 
lot of introductory material we assume the reader is already familiar with the basics of 
classical model theory (see Poizat |Poi85j ) and its generalisation to continuous logic (see 

[Bu]). 

Let T be a (classical or continuous) first order theory. 

Definition 5.1. We say that a formula (p(x,y) is dependent in T if for every A4 1= T, 
ip M is dependent on M n x M m . 

We say that T is dependent if all formulae are. 
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In the case of a classical theory this is equivalent to the original definition (see 
Laskowski |Las92j ) and it extends naturally to continuous logic. If T is a continuous 
dependent theory then by Lemma [2. 161 every definable predicate in T is dependent. In 
addition, it is easy to see using compactness that if <p(x,y) is dependent in T then it is 
uniformly so in all models of T. 

For a continuous language C let C R consists of a n-ary predicate symbol E[<^(x)] for 
every n-ary £-formula (p(x). 

Theorem 5.2. For every C-theory T (dependent or not) there is a (unique) C R -theory 
T R such that: 

(i) For every p[x) G S n (T R ) there is a unique Borel probability measure v v on S n (T R ) 
such that for every n-ary predicate symbol K[ip(x)] G C R : 



The mapping p i— > v v is a bijection between S n (T R ) and the space of regular 
Borel probability measures. We will consequently identify the two spaces, thus 
identifying p with v p . 

(ii) The topology on S n (T R ) is the one of weak convergence. In other words, this is 
the weakest topology such that for every continuous function ip: S n (T) — > C the 
mapping (U i-> J ipdfi is continuous. 

(iii) For a mapping f: m — > n, the corresponding mapping f* ,R : S n (T R ) — > S m (T R ) is 
given by associating to each type in S n (T R ), being a measure on S n (T), its image 
measure on S m (T) via the application /*: S n (T) — > S m (T). (Since /*: S„(T) — > 
S m (T) is continuous between compact spaces, the image of a regular measure is 
regular.) 

(iv) The distance predicate coincides with K[d(x,y)}. 

Moreover, T R eliminates quantifiers. 

Since every classical first order theory can be viewed as a continuous theory, the same 
applies if T is a classical theory. In this case we may prefer to write F[(p(x)} instead of 
E[(p(x)]. (In fact, the precise counterpart ofE[(p(x)] is F[-i(p(x)] since 1 is 'False", but 
this is a minor issue.) In particular the distance predicate is then given by F[x ^ y]. 

Proof. Uniqueness follows from the fact that the type spaces are entirely described. 

In the case T is a classical theory, the explicit construction appears in [BK] . where 
Keisler's original construction |Kei99| is transferred from classical logic to the more ade- 
quate setting of continuous logic. 

A similar construction can in principle be carried out when T is a continuous theory. 
Alternatively, let us consider S n (T R ) as a mere symbol denoting the space of regular Borel 
probability measures on S n (T). Let S(T R ) denote the mapping n \— > S n (T R ) and let us 
equip it with the topological and functorial structure described in items (ii),(iii). Then 
S(T R ) is an open Hausdorff type-space functor in the sense of |Ben03| . The predicates 




p — 
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of C R can be interpreted in models of S(T R ) as per item (i), in which case K[d(x, y)} 
defines a metric on the models. By results appearing in |BUj a continuous theory T R 
exists in some language whose type space functor is S n (T R ). Since the n-ary /^-formulae 
are dense among all continuous functions S n (T) — > [0, 1], the atomic £ R -formulae E[<^(a;)] 
separate types. It follows that T R can be taken to be an £ R -theory and that it eliminates 
quantifiers as such. We leave the details to the reader. Bs.2 

Members of models of T R should be thought of as random variables in models of T. If 
a, b, . . . G M. 1= T R then their type tp iJ (a, b, . . .), viewed as a probability measure, should 
be thought of as the distribution measure of the S n (T)-valued random variable ui \— > 
tp(a(o;), b(u), . . .). Similarly E[</?(a, b, . . .)] is the expectation of the random variable 
uj i — ► if(a(uj),b(uj), . . .), and so on. As we said in the introduction it is natural to ask 
whether the randomisation of a dependent theory is dependent. 

Theorem 5.3. Let T be a dependent first order theory (classical or continuous). Then 
T R is dependent as well. 

Proof. Every classical theory can be identified with a continuous theory via the identi- 
fication of T with 0, of F with 1 and of = with d. We may therefore assume that T is 
continuous. 

Let us first consider a formula of the form <p(x, y) = K[tp(x, y)]. Let M. N T R , and we 
need to show that ip M is dependent on M n x M m . Let us enumerate M n = {di \ i G J}, 
M m = {If. j e J}. Let p = tp(M n ,M m /0). We may write it as p{x h yj) i&I ,j&j e 
Siuj(T r ), and identify it with a probability measure /ionO = S(ix n )u(Jxm)(T) such that 
for every formula p(z) of the theory T, z C {xi,yj} ieI j e j: 



For i G /, j G J and q G Q define: Xqihj) = ^(^iyVj) 9 - Then \ = {Xq : ? G fi} is a 
measurable family of [0, l]-valued functions on fx J and (p(d~i, bj) = <p(xi, y^) v = ~E[x](i,j) 
where expectation is with respect to \i. Since T is dependent the family {x q - q G Q} is 
uniformly dependent. By Corollary 14.21 E[y] : I x J — > [0, 1] is dependent. Equivalently, 
if: M n x M m -> [0, 1] is dependent. 

We have thus shown that every atomic formula is dependent. By Lemma 13.61 every 
quantifier free formula is dependent. By quantifier elimination and Lemma 12.161 every 
formula is dependent. Bs.3 

We conclude this paper with a few extensions of classical results regarding dependent 
formulae and theories to continuous logic. 

Lemma 5.4. The following are equivalent for a formula (p(x, y): 
(i) The formula ip is independent. 
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(ii) There exist a tuple a an, indiscernible sequence (b n : n < uj) and < r < s < 1 
such that: 

(p(a,b 2n ) < r, <p(a,b 2n+1 ) > s. 

(iii) There exist a tuple a and indiscernible sequence (b n : n < uj) such that \imip(a, b n ) 
does not exists. 

Proof. (i) (ii). Assume ip is independent, and let us work in a sufficiently satu- 
rated model. Then there are < r < s < 1 such that for all m there are (b n : n < m) 
and {a w : w C m) satisfying: 

^(flw, K) < r new, (p(a w , b n ) > s <=^> n<£w. 

By compactness there exists an infinite sequence (b n : n < uj) such that for every finite 
u C uj and every w C u there are a UtW such that for all n G u: 

tp(a u ,w, K) < r new, <p(a U;W , b n ) > s <=^> n£w. 

By standard arguments using Ramsey's Theorem there exists an indiscernible sequence 
{b n : n < uj) having the same property. In particular for every m there exists a rn such 
that for all n < m: 

(p{a,b 2n ) < r, ^>{a,b 2n+1 ) > s. 

The existence of a as desired now follows by compactness. 

(ii) =^> (iii). Immediate. 

(iii) (i). Assume that (b n : n < uj) is indiscernible and lim n (p(a, b n ) does not exist. 
Then there are < r < s < 1 such that (p(a,b n ) < r and (p(a,b n ) > s infinitely often. 
Then for every m and every w C m we can find n < ... < n m -\ < uj such that 
tp(a, b ni ) < r if i e w and ip(a, b ni ) > s otherwise. By indiscernibility we can then find a w 
such that ip(a, bi) < r if i e w and <p(a, bi) > s if i e m\w. Then ip is independent. M 5A 

Lemma 5.5. Let a be a tuple, {b n : n < uj) an indiscernible sequence of tuples, and let 
Vsix,^. . - Vks-i) be dependent formulae for s e S. Then there exists in an elementary 
extension of M. an a-indiscernible sequence (c n : n < uj) such that for all s e S: 

(p s (a,c . ■ .cjfc s _i) = \im.(p s (a,b n . . .b n+ks -i). 

Proof. For k < uj let Ik consist of all increasing tuples n e uj h . We define a partial ordering 
on Ik saying that n < n' if nk-i < n' . Then standard arguments using Ramsey's Theorem 
and compactness yield an a-indiscernible sequence (c n : n < uj) such that for every k and 
every formula p(x,y , . . . ,y k -i)- 

lim<^(a,^) < (p(a,co,...,Ck-i) < lim ip(a, b n ), 

ndl k ™ e/ fe 

where b n = b m ,.. ■ ,K k _ x . 

Let us now fix s e S. If (n m : m < uj) is an increasing sequence in Ik s then {b nm : m < uj) 
is an indiscernible sequence so lim m p s {a, bn m ) exists. Moreover, given two increasing 
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sequences in we can choose a third increasing sequence alternating between the two, 
so the limit does not depend on the choice of sequence. It follows that lim neIk ip s (a,b n ) 
exists, and the assertion follows. M 5 5 

Theorem 5.6. Assume T is independent. Then there exists a formula ip(x,y), where x 
is a singleton, which is independent. 

Proof. Let (p(x, y) be an independent formula such that x has minimal length. If it is of 
length one we are done. If not, we may write x = xz and ip = tp(xz, y). 

By Lemma 15.41 there are ah and a sequence (c n : n < uj) is a model of T as well as 
< r < s < 1 such that 

<p(ab, c 2n ) < r, <p{ab, c 2n +i) > s. 

Choosing t G (r, s) dyadic and replace (p with m(ip — t) for m large enough we may assume 
that r = and s — 1. For m < uo let: 

ij) m (z, y , ■ ■ ■ , V2m-i) = inf V iv{xz, y 2 i) V ^(p(xz, y 2i+ i))- 

x » 

i<m 

By assumption of minimality of x the formulae ip n must be dependent. By Lemma [5.51 
there is a 6-indiscernible sequence (c'„ : n < uj) such that for all m: 

(2) ip m {b, c' . . . c 2m _i) = \imip m (b, 

We know that if) m {b, c 2ril . . . , C2 n +2m-i) = 0, as this is witnessed by a. Therefore the limit 
in ([2]) must be equal to zero, and thus ip m {b, c' Q . . . c 2m -\) = for all m. By a compactness 
argument there exists a' such that 

¥( a % 4J = 0. c' 2n+1 ) = 1. 

Changing our point of view a little we observe that {bc' n : n < ui) is an indiscernible 
sequence and 

(p(a', bc' 2n ) = 0, (p(a', bc' 2n+1 ) = 1. 
Thus (p(x, zy) is independent and x was not minimal after all. Bs.6 
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